重要性采样（IS）是非政策评估中的一种流行技术，它重新赋予了重播缓冲液中轨迹的回归以提高样本效率。但是，对IS进行培训可能是不稳定的，以前试图解决此问题的尝试主要集中于分析IS的差异。在本文中，我们揭示了不稳定性与IS的重复使用偏见的新概念有关 - 由重复使用缓冲液重用进行评估和优化引起的非政策评估偏差。从理论上讲，我们证明了对当前策略的非政策评估和优化，并通过重播缓冲区的数据导致目标高估，这可能会导致错误的梯度更新并退化性能。我们进一步提供了重复使用偏差的高概率上限，并表明控制上限的一个项可以通过引入非政策算法的稳定性概念来控制重复使用偏置。基于这些分析，我们最终提出了一种新颖的偏见调查重要性抽样（BIRIS）框架以及实际算法，可以减轻重复使用偏见的负面影响。实验结果表明，我们基于BIRIS的方法可以显着提高一系列连续控制任务的样品效率。

视觉导航中体现的代理以及深度神经网络引起了越来越多的关注。但是，深层神经网络容易受到恶意的对抗噪声的影响，这可能会导致视力导航的灾难性失败。在这些对抗性噪声中，通用的对抗扰动（UAP），即代理接收到的每个帧应用的图像无关扰动，对于体现视觉导航而言更为重要，因为它们是攻击过程中计算效率和应用程序实行的。但是，现有的UAP方法不考虑具体视觉导航的系统动力学。为了在连续决策设置中扩展UAP，我们将Universal Noise $ \ delta $下的不受欢迎的环境制定为$ \ delta $ distant的马尔可夫决策过程（$ \ delta $ -MDP）。基于该公式，我们分析了$ \ delta $ -MDP的性质，并提出了两种新型的一致攻击方法，用于攻击体现剂，它们首先通过估计受干扰的Q函数和干扰分布来考虑MDP的动态。尽管有受害者模型，但我们一致的攻击可能会导致栖息地目标任务的绩效大大下降。广泛的实验结果表明，将具体视觉导航方法应用于现实世界中存在潜在的风险。

尽管深度强化学习（DRL）取得了巨大的成功，但由于过渡和观察的内在不确定性，它可能遇到灾难性的失败。大多数现有的安全加固学习方法只能处理过渡干扰或观察障碍，因为这两种干扰影响了代理的不同部分。此外，受欢迎的最坏情况可能会导致过度悲观的政策。为了解决这些问题，我们首先从理论上证明了在过渡干扰和观察障碍下的性能降解取决于一个新颖的价值函数范围（VFR），这与最佳状态和最坏状态之间的价值函数的间隙相对应。基于分析，我们采用有条件的价值风险（CVAR）作为对风险的评估，并提出了一种新颖的强化学习算法的CVAR-Proximal-Policy-oftimization（CPPO），该算法通过保持风险敏感的约束优化问题形式化。它的CVAR在给定的阈值下。实验结果表明，CPPO获得了更高的累积奖励，并且在Mujoco中一系列连续控制任务上的观察和过渡干扰更加强大。

深增强学习模型容易受到对抗的攻击，可以通过操纵受害者的观察来减少受害者的累积预期奖励。尽管以前的优化基于优化的方法效率，用于在监督学习中产生对抗性噪声，因此这些方法可能无法实现最低的累积奖励，因为它们通常不会探索环境动态。在本文中，我们提供了一个框架，以通过重新制定函数空间中加固学习的对抗攻击问题来更好地了解现有方法。我们的重构在有针对性攻击的功能空间中产生最佳对手，通过通用的两级框架来排斥它们。在第一阶段，我们通过黑客攻击环境来培训欺骗性政策，并发现一组轨迹路由到最低奖励或最坏情况性能。接下来，对手误导受害者通过扰乱观察来模仿欺骗性政策。与现有方法相比，我们理论上表明我们的对手在适当的噪声水平下更强大。广泛的实验展示了我们在效率和效力方面的优越性，在Atari和Mujoco环境中实现了最先进的性能。

随着社交软件和多媒体技术的持续发展，图像已成为传播信息和社交的重要载体。如何全面评估图像已成为最近研究的重点。传统的图像美学评估方法通常采用单个数值总体评估评分，该评估具有一定的主观性，无法再满足更高的美学要求。在本文中，我们构建了一个称为Aesthetic混合数据集的新图像属性数据集，该数据集具有属性（AMD-A）和设计融合的外部属性功能。此外，我们还提出了一种有效的方法，用于在混合多属性数据集上进行图像美学属性评估，并通过使用ExtisticNet-B0作为骨干网络来构建多任务网络体系结构。我们的模型可以实现美学分类，整体评分和属性评分。在每个子网络中，我们通过ECA通道注意模块改进特征提取。至于最终的整体评分，我们采用了教师学习网络的想法，并使用分类子网络来指导美学的整体细粒回归。实验结果，使用思维螺旋式的结果表明，我们提出的方法可以有效地改善美学整体和属性评估的性能。

Benefiting from the intrinsic supervision information exploitation capability, contrastive learning has achieved promising performance in the field of deep graph clustering recently. However, we observe that two drawbacks of the positive and negative sample construction mechanisms limit the performance of existing algorithms from further improvement. 1) The quality of positive samples heavily depends on the carefully designed data augmentations, while inappropriate data augmentations would easily lead to the semantic drift and indiscriminative positive samples. 2) The constructed negative samples are not reliable for ignoring important clustering information. To solve these problems, we propose a Cluster-guided Contrastive deep Graph Clustering network (CCGC) by mining the intrinsic supervision information in the high-confidence clustering results. Specifically, instead of conducting complex node or edge perturbation, we construct two views of the graph by designing special Siamese encoders whose weights are not shared between the sibling sub-networks. Then, guided by the high-confidence clustering information, we carefully select and construct the positive samples from the same high-confidence cluster in two views. Moreover, to construct semantic meaningful negative sample pairs, we regard the centers of different high-confidence clusters as negative samples, thus improving the discriminative capability and reliability of the constructed sample pairs. Lastly, we design an objective function to pull close the samples from the same cluster while pushing away those from other clusters by maximizing and minimizing the cross-view cosine similarity between positive and negative samples. Extensive experimental results on six datasets demonstrate the effectiveness of CCGC compared with the existing state-of-the-art algorithms.

As one of the prevalent methods to achieve automation systems, Imitation Learning (IL) presents a promising performance in a wide range of domains. However, despite the considerable improvement in policy performance, the corresponding research on the explainability of IL models is still limited. Inspired by the recent approaches in explainable artificial intelligence methods, we proposed a model-agnostic explaining framework for IL models called R2RISE. R2RISE aims to explain the overall policy performance with respect to the frames in demonstrations. It iteratively retrains the black-box IL model from the randomized masked demonstrations and uses the conventional evaluation outcome environment returns as the coefficient to build an importance map. We also conducted experiments to investigate three major questions concerning frames' importance equality, the effectiveness of the importance map, and connections between importance maps from different IL models. The result shows that R2RISE successfully distinguishes important frames from the demonstrations.

Text clustering and topic extraction are two important tasks in text mining. Usually, these two tasks are performed separately. For topic extraction to facilitate clustering, we can first project texts into a topic space and then perform a clustering algorithm to obtain clusters. To promote topic extraction by clustering, we can first obtain clusters with a clustering algorithm and then extract cluster-specific topics. However, this naive strategy ignores the fact that text clustering and topic extraction are strongly correlated and follow a chicken-and-egg relationship. Performing them separately fails to make them mutually benefit each other to achieve the best overall performance. In this paper, we propose an unsupervised text clustering and topic extraction framework (ClusTop) which integrates text clustering and topic extraction into a unified framework and can achieve high-quality clustering result and extract topics from each cluster simultaneously. Our framework includes four components: enhanced language model training, dimensionality reduction, clustering and topic extraction, where the enhanced language model can be viewed as a bridge between clustering and topic extraction. On one hand, it provides text embeddings with a strong cluster structure which facilitates effective text clustering; on the other hand, it pays high attention on the topic related words for topic extraction because of its self-attention architecture. Moreover, the training of enhanced language model is unsupervised. Experiments on two datasets demonstrate the effectiveness of our framework and provide benchmarks for different model combinations in this framework.

An increasing number of public datasets have shown a marked clinical impact on assessing anatomical structures. However, each of the datasets is small, partially labeled, and rarely investigates severe tumor subjects. Moreover, current models are limited to segmenting specific organs/tumors, which can not be extended to novel domains and classes. To tackle these limitations, we introduce embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models, dubbed the CLIP-Driven Universal Model. The Universal Model can better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures. The model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from 3 datasets. We rank first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieve the state-of-the-art results on Beyond The Cranial Vault (BTCV). Compared with dataset-specific models, the Universal Model is computationally more efficient (6x faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks. The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting the previously learned classes.

Recent advances in self-supervised learning (SSL) in computer vision are primarily comparative, whose goal is to preserve invariant and discriminative semantics in latent representations by comparing siamese image views. However, the preserved high-level semantics do not contain enough local information, which is vital in medical image analysis (e.g., image-based diagnosis and tumor segmentation). To mitigate the locality problem of comparative SSL, we propose to incorporate the task of pixel restoration for explicitly encoding more pixel-level information into high-level semantics. We also address the preservation of scale information, a powerful tool in aiding image understanding but has not drawn much attention in SSL. The resulting framework can be formulated as a multi-task optimization problem on the feature pyramid. Specifically, we conduct multi-scale pixel restoration and siamese feature comparison in the pyramid. In addition, we propose non-skip U-Net to build the feature pyramid and develop sub-crop to replace multi-crop in 3D medical imaging. The proposed unified SSL framework (PCRLv2) surpasses its self-supervised counterparts on various tasks, including brain tumor segmentation (BraTS 2018), chest pathology identification (ChestX-ray, CheXpert), pulmonary nodule detection (LUNA), and abdominal organ segmentation (LiTS), sometimes outperforming them by large margins with limited annotations.